Scene text editing (STE) aims to replace text with the desired one while preserving background and styles of the original text. However, due to the complicated background textures and various text styles, existing methods fall short in generating clear and legible edited text images. In this study, we attribute the poor editing performance to two problems: 1) Implicit decoupling structure. Previous methods of editing the whole image have to learn different translation rules of background and text regions simultaneously. 2) Domain gap. Due to the lack of edited real scene text images, the network can only be well trained on synthetic pairs and performs poorly on real-world images. To handle the above problems, we propose a novel network by MOdifying Scene Text image at strokE Level (MOSTEL). Firstly, we generate stroke guidance maps to explicitly indicate regions to be edited. Different from the implicit one by directly modifying all the pixels at image level, such explicit instructions filter out the distractions from background and guide the network to focus on editing rules of text regions. Secondly, we propose a Semi-supervised Hybrid Learning to train the network with both labeled synthetic images and unpaired real scene text images. Thus, the STE model is adapted to real-world datasets distributions. Moreover, two new datasets (Tamper-Syn2k and Tamper-Scene) are proposed to fill the blank of public evaluation datasets. Extensive experiments demonstrate that our MOSTEL outperforms previous methods both qualitatively and quantitatively. Datasets and code will be available at https://github.com/qqqyd/MOSTEL.

Deep metric learning aims to learn an embedding space, where semantically similar samples are close together and dissimilar ones are repelled against. To explore more hard and informative training signals for augmentation and generalization, recent methods focus on generating synthetic samples to boost metric learning losses. However, these methods just use the deterministic and class-independent generations (e.g., simple linear interpolation), which only can cover the limited part of distribution spaces around original samples. They have overlooked the wide characteristic changes of different classes and can not model abundant intra-class variations for generations. Therefore, generated samples not only lack rich semantics within the certain class, but also might be noisy signals to disturb training. In this paper, we propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning. We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining and boost metric learning losses. Further, for most datasets that have a few samples within the class, we propose the neighbor correction to revise the inaccurate estimations, according to our correlation discovery where similar classes generally have similar variation distributions. Extensive experiments on five benchmarks show our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%. Our code is available at https://github.com/darkpromise98/IAA

Scene text spotting is of great importance to the computer vision community due to its wide variety of applications. Recent methods attempt to introduce linguistic knowledge for challenging recognition rather than pure visual classification. However, how to effectively model the linguistic rules in end-to-end deep networks remains a research challenge. In this paper, we argue that the limited capacity of language models comes from 1) implicit language modeling; 2) unidirectional feature representation; and 3) language model with noise input. Correspondingly, we propose an autonomous, bidirectional and iterative ABINet++ for scene text spotting. Firstly, the autonomous suggests enforcing explicitly language modeling by decoupling the recognizer into vision model and language model and blocking gradient flow between both models. Secondly, a novel bidirectional cloze network (BCN) as the language model is proposed based on bidirectional feature representation. Thirdly, we propose an execution manner of iterative correction for the language model which can effectively alleviate the impact of noise input. Finally, to polish ABINet++ in long text recognition, we propose to aggregate horizontal features by embedding Transformer units inside a U-Net, and design a position and content attention module which integrates character order and content to attend to character features precisely. ABINet++ achieves state-of-the-art performance on both scene text recognition and scene text spotting benchmarks, which consistently demonstrates the superiority of our method in various environments especially on low-quality images. Besides, extensive experiments including in English and Chinese also prove that, a text spotter that incorporates our language modeling method can significantly improve its performance both in accuracy and speed compared with commonly used attention-based recognizers.

There still remains an extreme performance gap between Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs) when training from scratch on small datasets, which is concluded to the lack of inductive bias. In this paper, we further consider this problem and point out two weaknesses of ViTs in inductive biases, that is, the spatial relevance and diverse channel representation. First, on spatial aspect, objects are locally compact and relevant, thus fine-grained feature needs to be extracted from a token and its neighbors. While the lack of data hinders ViTs to attend the spatial relevance. Second, on channel aspect, representation exhibits diversity on different channels. But the scarce data can not enable ViTs to learn strong enough representation for accurate recognition. To this end, we propose Dynamic Hybrid Vision Transformer (DHVT) as the solution to enhance the two inductive biases. On spatial aspect, we adopt a hybrid structure, in which convolution is integrated into patch embedding and multi-layer perceptron module, forcing the model to capture the token features as well as their neighboring features. On channel aspect, we introduce a dynamic feature aggregation module in MLP and a brand new "head token" design in multi-head self-attention module to help re-calibrate channel representation and make different channel group representation interacts with each other. The fusion of weak channel representation forms a strong enough representation for classification. With this design, we successfully eliminate the performance gap between CNNs and ViTs, and our DHVT achieves a series of state-of-the-art performance with a lightweight model, 85.68% on CIFAR-100 with 22.8M parameters, 82.3% on ImageNet-1K with 24.0M parameters. Code is available at https://github.com/ArieSeirack/DHVT.

文本到图像生成旨在生成与给定文本一致的真实图像。先前的作品主要通过堆叠生成器 - 歧义器对进行多个对抗训练，主要采用多阶段体系结构，在该培训中，用于提供发电指导的文本语义在所有阶段都保持静态。这项工作认为，每个阶段的文本特征应根据历史阶段的状态（即历史阶段的文本和图像特征）进行自适应重新组合，以在粗到精细的生成过程中提供多样化和准确的语义指导。因此，我们提出了一种新颖的动力学语义演化gan（DSE-GAN），以在新颖的单一对抗性多阶段体系结构下重新构成每个阶段的文本特征。具体而言，我们设计（1）动态语义演化（DSE）模块，该模块首先汇总了历史图像特征以总结生成反馈，然后动态选择在每个阶段重新组装的单词，并通过动态地组装它们增强或抑制不同的粒度子空间的语义。 （2）单个对抗性多阶段体系结构（SAMA），通过消除复杂的多个对抗训练要求扩展了先前的结构，因此可以允许更多的文本图像相互作用阶段，并最终促进DSE模块。我们进行了全面的实验，并表明DSE-GAN在两个广泛使用的基准分别（即CUB-200和MSCOCO）上获得了7.48 \％和37.8％的相对FID。

人类视频运动转移（HVMT）的目的是鉴于源头的形象，生成了模仿驾驶人员运动的视频。 HVMT的现有方法主要利用生成对抗网络（GAN），以根据根据源人员图像和每个驾驶视频框架估计的流量来执行翘曲操作。但是，由于源头，量表和驾驶人员之间的巨大差异，这些方法始终会产生明显的人工制品。为了克服这些挑战，本文提出了基于gan的新型人类运动转移（远程移动）框架。为了产生逼真的动作，远遥采用了渐进的一代范式：它首先在没有基于流动的翘曲的情况下生成每个身体的零件，然后将所有零件变成驾驶运动的完整人。此外，为了保留自然的全球外观，我们设计了一个全球对齐模块，以根据其布局与驾驶员的规模和位置保持一致。此外，我们提出了一个纹理对准模块，以使人的每个部分都根据纹理的相似性对齐。最后，通过广泛的定量和定性实验，我们的远及以两个公共基准取得了最先进的结果。

由于其广泛的应用，例如自动驾驶，机器人技术等，认识到Point Cloud视频的人类行为引起了学术界和行业的极大关注。但是，当前的点云动作识别方法通常需要大量的数据，其中具有手动注释和具有较高计算成本的复杂骨干网络，这使得对现实世界应用程序不切实际。因此，本文考虑了半监督点云动作识别的任务。我们提出了一个蒙版的伪标记自动编码器（\ textbf {Maple}）框架，以学习有效表示，以较少的注释以供点云动作识别。特别是，我们设计了一个新颖有效的\ textbf {de}耦合\ textbf {s} patial- \ textbf {t} emporal trans \ textbf {pert}（\ textbf {destbrof {destformer}）作为maple的backbone。在Destformer中，4D点云视频的空间和时间维度被脱钩，以实现有效的自我注意，以学习长期和短期特征。此外，要从更少的注释中学习判别功能，我们设计了一个蒙版的伪标记自动编码器结构，以指导Destformer从可用框架中重建蒙面帧的功能。更重要的是，对于未标记的数据，我们从分类头中利用伪标签作为从蒙版框架重建功能的监督信号。最后，全面的实验表明，枫树在三个公共基准上取得了优异的结果，并且在MSR-ACTION3D数据集上以8.08 \％的精度优于最先进的方法。

知识图（kgs）在许多应用程序中越来越重要的基础架构，同时患有不完整问题。 KG完成任务（KGC）自动根据不完整的KG预测缺失的事实。但是，现有方法在现实情况下表现不佳。一方面，他们的性能将巨大的降解，而kg的稀疏性越来越大。另一方面，预测的推理过程是一个不可信的黑匣子。本文提出了一个稀疏kgc的新型可解释模型，将高阶推理组合到图形卷积网络中，即HOGRN。它不仅可以提高减轻信息不足问题的概括能力，而且还可以在保持模型的有效性和效率的同时提供可解释性。有两个主要组件无缝集成以进行关节优化。首先，高阶推理成分通过捕获关系之间的内源性相关性来学习高质量的关系表示。这可以反映逻辑规则，以证明更广泛的事实是合理的。其次，更新组件的实体利用无重量的图形卷积网络（GCN）有效地模拟具有可解释性的KG结构。与常规方法不同，我们在没有其他参数的情况下在关系空间中进行实体聚合和基于设计组成的注意。轻巧的设计使HOGRN更适合稀疏设置。为了进行评估，我们进行了广泛的实验 - HOGRN对几个稀疏KG的结果表现出了令人印象深刻的改善（平均为9％的MRR增益）。进一步的消融和案例研究证明了主要成分的有效性。我们的代码将在接受后发布。

视频的行动识别，即将视频分类为预定义的动作类型之一，一直是人工智能，多媒体和信号处理社区中的一个流行话题。但是，现有方法通常考虑一个整体上的输入视频并学习模型，例如卷积神经网络（CNNS），并带有粗糙的视频级别类标签。这些方法只能为视频输出一个动作类，但不能提供可解释的线索来回答为什么视频显示特定的动作。因此，研究人员开始专注于一项新任务，部分级别的动作解析（PAP），该作用不仅旨在预测视频级别的动作，而且还要认识到每个人的框架级别的细粒度的动作或身体部位的相互作用在视频中。为此，我们为这项具有挑战性的任务提出了一个粗到精细的框架。特别是，我们的框架首先预测输入视频的视频级别类别，然后将身体部位定位并预测零件级别的动作。此外，为了平衡部分级别的动作解析的准确性和计算，我们建议通过段级特征识别零件级的操作。此外，为了克服身体部位的歧义，我们提出了一种姿势引导的位置嵌入方法来准确地定位身体部位。通过在大规模数据集（即动力学TPS）上进行的全面实验，我们的框架可以实现最先进的性能，并且超过31.10％的ROC得分的现有方法。

使用神经网络代表3D对象已变得流行。但是，许多以前的作品采用具有固定体系结构和大小的神经网络来表示不同的3D对象，这导致简单对象的网络参数过多，并且对复杂对象的重建精度有限。对于每个3D模型，希望拥有尽可能少的参数以实现高保真重建的端到端神经网络。在本文中，我们提出了一种利用神经体系结构搜索（NAS）和二进制分类的高效体素重建方法。以层数，每一层的节点数量以及每一层的激活函数为搜索空间，可以根据强化学习技术获得特定的网络体系结构。此外，为了摆脱网络推理后使用的传统表面重建算法（例如，行进立方体），我们通过对二进制体素进行分类来完成端到端网络。与其他签名的距离字段（SDF）预测或二进制分类网络相比，我们的方法使用更少的网络参数获得了更高的重建精度。